SherbyNet - /page/wikibook/entry.mu

Micron Document

CuPy

18.2 KB · ⇊ 11 parts · ⤓ Micron 18.2 KB · ⤓ HTML 57.0 KB

──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

top

CuPy is an open source library for GPU-accelerated computing with Python programming language, providing support for multi-dimensional arrays, sparse matrices, and a variety of numerical algorithms implemented on top of them.cite-ref-3[3] CuPy shares the same API set as NumPy and SciPy, allowing it to be a drop-in replacement to run NumPy/SciPy code on GPU. CuPy supports Nvidia CUDA GPU platform, and AMD ROCm GPU platform starting in v9.0.cite-ref-4[4]cite-ref-5[5]

CuPy has been initially developed as a backend of Chainer deep learning framework, and later established as an independent project in 2017.cite-ref-6[6]

CuPy is a part of the NumPy ecosystem array librariescite-ref-7[7] and is widely adopted to utilize GPU with Python,cite-ref-8[8] especially in high-performance computing environments such as Summit,cite-ref-9[9] Perlmutter,cite-ref-10[10] EULER,cite-ref-11[11] and ABCI.cite-ref-12[12]

CuPy is a NumFOCUS sponsored project.cite-ref-13[13]

Contents

• Features

• NumPy-compatible APIs

• SciPy-compatible APIs

• User-defined GPU kernels

• Distributed computing

• Low-level CUDA features

• Interoperability

• Examples

• Array creation

• Basic operations

• Raw CUDA C/C++ kernel

• Applications

• See also

• References

• External links

Features

CuPy implements NumPy/SciPy-compatible APIs, as well as features to write user-defined GPU kernels or access low-level APIs.cite-ref-14[14]cite-ref-15[15]

NumPy-compatible APIs

The same set of APIs defined in the NumPy package (numpy.*) are available under cupy.* package.

• Multi-dimensional array (cupy.ndarray) for boolean, integer, float, and complex data types

• Module-level functions

• Linear algebra functions

• Fast Fourier transform

• Random number generator

SciPy-compatible APIs

The same set of APIs defined in the SciPy package (scipy.*) are available under cupyx.scipy.* package.

• Sparse matrices (cupyx.scipy.sparse.*_matrix) of CSR, COO, CSC, and DIA format

• Discrete Fourier transform

• Advanced linear algebra

• Multidimensional image processing

• Sparse linear algebra

• Special functions

• Signal processing

• Statistical functions

User-defined GPU kernels

• Kernel templates for element-wise and reduction operations

• Raw kernel (CUDA C/C++)

• Just-in-time transpiler (JIT)

• Kernel fusion

Distributed computing

• Distributed communication package (cupyx.distributed), providing collective and peer-to-peer primitives

Low-level CUDA features

• Stream and event

• Memory pool

• Profiler

• Host API binding

• CUDA Python supportcite-ref-16[16]

Interoperability

• DLPackcite-ref-17[17]

• CUDA Array Interfacecite-ref-18[18]

• NEP 13 (__array_ufunc__)cite-ref-19[19]

• NEP 18 (__array_function__)cite-ref-20[20]cite-ref-21[21]

• Array API Standardcite-ref-22[22]cite-ref-23[23]

Examples

Array creation

>>> import cupy as cp

>>> x = cp.array([1, 2, 3])

>>> x

array([1, 2, 3])

>>> y = cp.arange(10)

>>> y

array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

Basic operations

>>> import cupy as cp

>>> x = cp.arange(12).reshape(3, 4).astype(cp.float32)

>>> x

array([[ 0., 1., 2., 3.],

[ 4., 5., 6., 7.],

[ 8., 9., 10., 11.]], dtype=float32)

>>> x.sum(axis=1)

array([ 6., 22., 38.], dtype=float32)

Raw CUDA C/C++ kernel

>>> import cupy as cp

>>> kern = cp.RawKernel(r'''

... extern "C" __global__

... void multiply_elemwise(const float* in1, const float* in2, float* out) {

... int tid = blockDim.x * blockIdx.x + threadIdx.x;

... out[tid] = in1[tid] * in2[tid];

... }

... ''', 'multiply_elemwise')

>>> in1 = cp.arange(16, dtype=cp.float32).reshape(4, 4)

>>> in2 = cp.arange(16, dtype=cp.float32).reshape(4, 4)

>>> out = cp.zeros((4, 4), dtype=cp.float32)

>>> kern((4,), (4,), (in1, in2, out)) # grid, block and arguments

>>> out

array([[ 0., 1., 4., 9.],

[ 16., 25., 36., 49.],

[ 64., 81., 100., 121.],

[144., 169., 196., 225.]], dtype=float32)

Applications

• spaCycite-ref-24[24]cite-ref-25[25]

• XGBoostcite-ref-26[26]

• turboSETI (Berkeley SETI)cite-ref-27[27]

• NVIDIA RAPIDScite-ref-28[28]cite-ref-29[29]cite-ref-30[30]cite-ref-31[31]

• einopscite-ref-32[32]cite-ref-33[33]

• scikit-learncite-ref-34[34]

• MONAI

• Chainercite-ref-35[35]